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ABSTRACT 


A computer program was written that carries out reliabi- 
lity assessments according to a method proposed by the TRW 
Corporation. This method combines point estimates for 
reliability from different sources into an overall point 
estimate. The vrogram was used to calculate the overall 
point estimate for cases covering a range of sample sizes, 
and underlying probabilities of success in order to make a 


judgement on the usefulness of the method. 
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Lo. ENTRODUCTION 


In deriving a point estimate for the reliability of a 
system or component, there is no universally accepted method 
for combining data from more than one test environment. For 
example, if a system on a test stand performs its intended 
function four times out of five trials, and in another 
environment, say actual use, it verforms its function in two 
out of three trials, the point estimates for system reliabi- 
lity in the two environments would be 4/5 and 2/3 respectively. 
It is not clear how these two estimates from different environ- 
ments should be combined into a single point estimate of 
realiabiltiy. 

The Quality Evaluation Laboratory, U.S. Naval Ammunition 
Depot, Oahu, Hawaii, published a procedure to accomplish this 
integration of trial data by comparing all data sets to a 
base data set and generating weighting factors for each data 
set. TRW Corporation wrote a computer routine to carry out 
the required computations for this method and proposed the 
method be adopted use in certain development programs. 

The TRW computer program was not available for this 
study and so two programs were written as part of this project. 
Each program has certain limitations in use, but together they 


cover a complete range of method parameters. 
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Lt. THE, METHOD 


Consider a series of n trials of a system of which m are 
judged successes. Then the probability that R is the under- 
lying reliability of the system is given by the binomial 


distribution. 
m -m 
PUR ee eRe a) Ook one 
This relationship generates a probability density function 


such as is shown in figure 1. This is the likelihood function 


of R given m successes inn trials. 


P (R) FIGURE 1, Likelihood function 
of R given m successes in n 
te ials.. 


0 R | 
To evaluate percentiles of the binomial distribution, the 
incomplete beta function can be used through the following 
identity: 


n 
I, (a,n-atl) =o) iGaR (lek) > nema positive intégers. 
x=a 


line incomplete beta function is defined 


1 Oe amy b-1 


where 


Eta) i {b), 
T (a+b) 


Bla, bh = 
The incomplete beta family is extensively tabled and efficient 
computer routines have been written to compute I, (a,b) 
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If the probability density function for a second set of 
test data is superimposed on the first density function, a 


representation such as figure 2 results. 


FIGURE 2. Two likelihood 
functions showing overlap. 


P (R) 
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The TRW method uses the amount of overlap as a weighting 
factor that measures the extent of agreement between the two 
sets of data. Calculating the weighting factor can be done 
by first solving for points of intersection of the two curves. 


These are all points x which are solutions to the equation 


sche (tex) = Ce x> (1-x)P7S 
where 
n = number of trials in data set l 
m = number of successes in data set l 
p = number of trials in data set 2 


Ss = number of successes in data set 2 


This expression simplifies to 


m-S)5_..0-n=pts .- pl (nem) tom! 
= so ~ -(p=s)) S) nl 


This is a polynomial of degree n-p. 

With the points of intersection determined, the TRW 
method determines the weighting factor by summing the area 
under that curve which is the lower between each intersection. 


See figure 2. 
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The two computer programs included in this report perform 
these calculations. Limitations on the programs are discussed 
in the descriptions that introduce each program. 

To calculate a point estimate of reliability using the 
few method, each set of data; i = 1,2;...;n,. 1S Compared wrth 
a set of base data and a weighting factor, key is determined 
as discussed above. These weighting factors are then applied 


to the point estimates to determine the overall estimate as 


follows: 
y 1.0(PY) + kj (Pj) + k, (P35) “Pataca Sr k (PS) 
: £0) + ky + ky oo kn 
where 
R = overall point estimate of reliability 
Ps = point estimate from 4 Eh data set 


Py = point estimate from base data set 


k. = weighting factor from Bee data set 


2 = ee ecoe Kh 


III. ANALYSIS OF TRW METHOD 


The magnitude of the weighting factor or overlap used in 
this method is determined by the relationship between the two 
incomplete beta functions used to represent the data sets. 
Specifically, the SGSilEe is,a function of (1) the Vocation 
of the mean of each curve relative to the mean of the other 
end and (2) the spread or variance of each curve.+ A parti- 
cular realization of the beta function tends to become spike 
shaped as the number of trials is increased. This effect is 
shown in table 1 which lists by way of example seven data 


sets, all having a mean of .8000. 


Number of trials Beta curve Variance 

9 B(x;5,4) .0246 

18 B(x;10,8) 40129 

25 Bix3' 20); 5) .0062 

40 B(x 32.8) .0039 

50 B(x;40,10) . 0031 

60 B(x; 48,12) .0026 

#100 B(x;80,20) -0016 


Table 1. As the number of trials is increased, the 
variance of the beta distribution representation decreases. 
“The means of all data sets is .8000. 

It was hypothesized that the weighting factor would be rela- 


tively small (and consequently the TRW method would have only 


1 ‘ ' : ; ‘ 
The mean and variance of the beta distribution representing 


a data set with s sulccesses and £ farlures are 


Ss : Se 
mean = ——— Variance = ————— 
(st+f) 
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a minor effect on reliability calculations) when (1) the 
ratios of successes to total trials from each data set were 
widely displaced from each other, or (2) when the number of 
trials in either data set was not small and the ratio of 
successes to number of trials were more than a little displaced 
from each other. 

To investigate this idea, pairs of test data sets and 
base data sets were compared and a weighting factor and point 
estimate of reliability was determined using the TRW method 
and the computer routines written for this project. A range 
of probabilities of success on a single trial and total 
number of trials was considered for both base and test data 
sets. Thirty six pairs were considered. 

Table 2 is a tabulation of the data from the investigation. 
The number of trials in the base set, Nye and in the trial 


data set, are shown for each case as are the underlying 


Nyy 
probabilities of success on a Single trial, Py for the base 


data and Pe for the trial data. Fach of the ten replications 


using these parameters resulted in a number of successes, Sp 


and Sys and point estimates of reliability, Py and Pie These 


last were calculated as 


The number of successes and the number of failures in each 
replication were inputs into the computer program which 


generated the TRW weighting factor, K. The TRW point estimate 
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of reliability, R, was then calculated as 


nel MeO tz Py + Kx Pr 
a LQ. se 4K ; 


The magnitude of the adjustment to the base data point esti- 
mate is the absolute value of the difference R-Py. The 
values generated in the ten replications were averaged and 


these average values, Spr Py, Sy, Pp? 


a Ky. JBaawand [R=Py |, are 


the entries in table 2. 

By way of example, consider case 1. In this case, the 
base data set consisted of ten trials (ny) each with a 
probability of success of .99 (Py). The test data set con- 
sisted of twenty trials (ni), each with a probability of success 
of .80 (P,). The results of each of the ten replications are 
shown below in table 3. (Data from all replications is con- 


tained in the computer output section of this report.) 


Replication Sy Py Sy Pe TRW K R |R-P, | 
a 9 .90 16 80 .5024 .8666 .0334 

2 9 .90 16> 80 .5024 .8666 .0334 

3 9 .90 17 «85 .6641 ,8800. 20200 

4 9 .90 1? 485 .6641 .8800 .0200 

5 9 90 17-385 .6641 .8800 .0200 

6 9 .90 sy .6641 .8800 .0200 

7 9 .90 1g 790 .8673 .9000 .0000 

8 9 90 ie +90 .8673 .9000 .0000 

9 9 .90 PS 65 2019. «8580 .0420 

10 9 90 15 6.75 a7. J6592.7 30408 
Averages 9.0 590 1654) “282 O97 “LETTE e229 


Table 3. Example showing the generation of data for 


Table 2, Case ls Base Data Set: n. = 10, PP. = 299, Test 


b b 


Data Sets. n, = 20, 2. = .60: 
The last row in table 3 shows the average figures that are 
the table 2 entries for case 1. Table 2 also lists the standard 


deviation of the distribution of R for each case. 
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The data of table 2 shows that the TRW method consistently 
makes a correction to the point estimate of reliability that 
is in the proper direction. The largest correction to the 
point estimate in any of the 360 computations was .042. 

This occurred when the base data was nine successes out of 
ten trials and the test data was thirteen successes out of 
twenty trials. As hypothesized, as the number of trials in 
either data set inerenued, or as the point estimates from 

the data sets diverged, the correction to the point estimate 
decreased. Whenever the point estimates coincided, there was 


no correction. 


i3 


ee 


- 


IV. CONCLUSIONS 


The correction that the TRW method applies to a base data 
point estimate decreases as the number of trials in either 
data set increases or as the point estimates from the data 
sets diverge. The method probably would adjust the point 
estimate less than would an observer making a subjective 


judgement based on the data. 
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